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SIGNAL PROCESSING BY MEANS OF RESONATORS 
TECHNICAL FIELD 

The invention concerns signal processing systen^ comprising resonator arrays and schemes 
5 for signal processing. 

BACKGROUND OF THE INVENTION 

The market of computing expands rapidly in pervasive computing devices (also referred to 
as tier-O devices). As these devices will be part of almost every aspect of our life, and as the 
functions of these devices will become increasingly complex, it is mandatoty that the 
10 interaction between man and machine mimics the ways humans interact with each other. 

Thus there is a need of simple signal processing techniques, e,g., for the processing of voice 
or other audio signals. 

There are many other areas where there is a demand for better signal processing systems. 
Examples are hearing aids, speaker recognition systatis, noise suppression systems, and 
15 systems that allow a speaker to control certain functions of a computer, device, vehicle, 
machine, or apparatus, and so forth. 

Signal processing typically involves filtering of an input signal in the frequency domain. In 
the standard appoach, the input signal is first transfortned into the frequency domain using 
well known Fast Fourier Transform (FFT) algorithms. In the frequency domain, 

20 implementation of the filter opemtion is accomplished by a multiplication of the filter 

response function with the transformed input signal- A filtered time domain output signal is 
then recovered by moms of a subsequent inverse FFT operation. Although extremely 
efficient algorithms are known for implementing this approach it suffers from a number of 
fundamental deficiencies. The FFT transformation is a non-local operation in the time 

25 domain. This means that an intrinsic time delay is introduced which is inversely 

proportional to the frequency resolution of the transformation. In order to avoid disturbing 
echoing effects the time delay should typically not exceed 10"^ s in audio applications which 
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corresponds to a frequency resolution of 100 Hz, at best. In order to achieve sufficient 
frequency resolution for a satisfactory synthesis of the filtered signal one must resort to 
complex phase analysis methods which substantially increase the computational effort. 

Standard signal processing techniques are computationally expensive and are thus not 
suited for use in voice recognition or speaker recognition as implemented in some tear-0 
devices such as smart cards. 

Similar problems occur in today's hearing aids, where specified spectral regions are 
amplified- For these applications ttie time delay is particular detrimental, because here the 
original and the processed signal overlap, and thus smear out the subtle time infonnation. 

It is a disadvantage of known systems that the time resolution is not hi^ enough to provide 
reliable cues for use by speech recognition systems. 

It is an object of the present invention to provide an improved signal processing approach. 

It is an object of the present invention to provide an improved signal processing approach 
which is suited for use in hearing aids. 

It is an object of the present invention to provide an improved signal processing approach 
for use in speech recognition or speaker recognition systems. 

It is an object of the present invention to provide an improved signal processing approach 
for processing acoustic signals. 
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SUMMARY OF THE INVENTION 

The present invention concerns a method, signal processing system, a computer program 
element, and a computer program product as claimed in the claims. 

Advantages of the present invention are either explicitly addressed in the detailed 
description or are obvious from the disclosed* 
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DESCRIPTION OF THE DRAWINGS 

The invention is described in detail below with reference to the following scheniatic 
drawings. It is to be noted that the Figures are not drawn to scale. 

FIG* lA is a schematic block diagram of an embodiment, in accordance with the 
present invention. 

FIG. IB is a schematic block diagram of another embodiment, in accordance with the 
present invention. 

FIG* iC is a schematic block diagram of another embodiment, in accordance with the 
present invention. 

FIG* 2 is a schematic flow chart that is used to illustrate aspects of the present 
invention. 

FIG* 3 shows the hearing threshold of an impaired ear relative to normal hearing. 

FIG* 4 is a schematic of loudness growth in normal (solid line) and imj^ired (dashed 
line) ears used to illustrate recruitment. Pmax denotes the intercept point for 
equal loudness perception of normal and unpaired hearing. 

FIG* 5 is a schematic that shows the adjustment of the amplification in a hearing aid, 
according to the present invention, depending on the spectral sound intensity. 

(a) Amplification vs. frequency is equal to the hearing threshold curve of the 
impaired ear for a white noise input spectrum with an intensity of 0 dB 
(hearing threshold for normal hearing). 

(b) Same as (a) but now the input intensity is P - 50 dB corresponding to the 
loudness of a normal conversation. To account for recruitment the 
amplification curve must be re-scaled by a factor of ( 1 - P/Pn&x = 0.5) 

(c) The spectral intensity of real sound signals is non-uniform. 
Correspondingly, the scale factor is frequency dependent giving the 
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amplification curve an entirely different shape from the original threshold 
curve. 

BIG* 6 Is a schematic drawing of an embodiment of a hearing aid in accordance with 
the present invention. 

5 FIG, 7 Shows exemplary elements of the unit 6 1 i in Figure 6. 

FIG, 8 Is an alternative embodiment of a hearing aid using a set of pseudo-orthogonal 
basis functions which are derived from the resonator transforms by means of a 
linear transformation* 

HG, 9 Is a schematic block diagram of an embodiment of a hearing aid in accordance 
10 with the present invention. 

BIG* 10 Is a schematic block cUagram of another embodiment of a hearing aid in 
accordance with the present invention. 

FIG. 11 Is a schematic extoiple of a general resonator in accordance with the present 
invention* 

15 FIG- 12 Is a schematic embodiment, in accordance with the present invention 

comprising a resonator array with a plurality of resonators as shown in Figure 
11. 

FIG, 13A Is a pseudo-code representation of a possible softwate implementation of a 
general damped (harmonic) resonator array in accordance with the present 
20 invention. 

FIG, 13B is a continuation of the pseudo-code representation of Figure 13A. 
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DESCRIPTION OF PREFERRED EMBODIMENTS: 

The basic concept of the present invention and &oi»e basic tenns and expressions are 
described in the following, 

A resonator in the most basic sense is a selective filter that damps or suppresses certain 
5 ftequcncies. A resonator can be described or characterized by its respoiise function. The 
response function describes the filter characteristic (the frequency dependent attenuation 
and phase shift) of a resonator. 

There are many ways to realize a resonator. A mechanical or micro-mechanical resonator 
for example might comprise a cavity or hollow body which vibrates syrxqmthetically with 
10 another vibrating system, such as the air through which an audio wave is traveling. The 
resonator can also bo implemented by means of micro-mechanical approaches using 
mechanical resonators such as cantilevers. 

Micromecharucal systems comprising cantilevers are well known and easy to make. 
Existing semiconductor techniques can be employed. In essence, the techniques of 

15 micromachining are employed to create cantilever arrays. Details on how to make such 
arrays are described in co-pending patent application PCT/IB95/00817 which was 
published on 10 April 1997 under publication number WO 97/13127 and in co-pending 
patent application PCT/IB96/00636 which was filed on 3 July 1996. The resonance 
frequency of each cantilever of the cantilever array is well defined such that a resonator 

20 array is obtamed which has resonators with well defined Q-factors and resonance 
frequencies. 

Bridges or membranes can be used instead of cantilevers. It is also conceivable to employ a 
combination of cantilevers, with bridges or membranes. 

If a micromechanical resonator array is to be used in connection with an elecuical signal 
25 processing circuitry (e.g. a reconslructor which assigns weights and superiK>ses the 
weighted signals), then the mechanical signals (vibration of the micromechanical 
resonators) has to be transformed into electrical signals. This can be done by means of 
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so-called detectors. Well suited are piezoelectric or piezoresistive detectors. Examples are 
given in above-mentioned co-pending PCT applications WO 97/13127 or 
PCT/IB96/00636. Likewise, well-known optical methods or capacitance sensing can be 
used to transform vibrational movement into electrical signals. 

5 A mechanical resonator can also be realized using a system of springs. 

A resonator can be implemented by means of analog electronics using impedance, 
capacitors and resistors. In addition, active devices such as MOSFETs driven below 
switching threshold can be used to amplify signals. A resonator can also be realized by a 
filter that blocks high frequencies and another filter that blocks low frequencies. If ftese 
10 two filters are coupled in a suited manner one obtains a filter response function which is a 
superposition of the response functions of the two filters. This filter response functicm 
might have a shape that is comparable to the shape of a resonator's response function. 

Likewise, resonattsrs can also be implemented in a computer system using ^me suited code 
that embodies algorithms. 

15 A resonator can also be realized as a combination of any of llie above-described 
approaches. 

All practical reson^ors show a loss of energy due to friction (internal or extemal) or other 
resistance. This is called damping. The resulting decrease in an^litude of the system is 
known as the decay. The term damping, then, is restricted to the system itself, mther than 
20 to the sound of the system in an environment 

Decay is the process whereby the amplitude of oscillation of a vibrating system (such as a 
resonator) diminishes with time due to a loss in energy. The loss of energy is what is 
usually called damping. The decay constant T (also known as decay time) is the time 
required for the amplitude of a vibrating system to decrease to approximately 37% (or 1/e) 
25 of its initial value. 
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A resonator - when set in motion - vibrates both as a whole, with a frequency called the 
fiindamental, and, with lesser intensity, in sections as well. If these smaller lengths are 
integer fractions (1/2, 1/3, 1/4, ,..), their frequencies of oscillation am called harmonics, and 
are integer multiples of the fundamental. Other resonating frequencies which are not whole 
5 multiples of the fundamental may also be present, and are called partials or inharmomc 
resonating frequencies. A resonator vibrates as a whole and damps or suppresses certain 
frequencies. That is, a resonator has some sort of a filter function. 

The present scheme makes use of the fact ihal an arbitrary time domain signal A(t) can be 
approximated by a linear superposition of response functions of resonators, and in 
10 particular I^rentzian resonators, with resonance frequencies and Q-factors chosen at will. 

The iTCsent scheme can be realized in a computing system such that tfie response functions 
aie generated by numerical integration of second order dif fetential equations which are 
associated with tiie resonators or Lorentzian resonators. This js numerically efficient and 
avoids problems which are encountered with the FFT approach, namely, time delays can be 
1 5 made negligible and tiie fiill spectral information is preserved, viz. there is no limit on the 
frequency resolution. Tte pi^sent scheme is rcal-time and first results are available right 
away. This is important for hearing aids, for example, where delay leads to echoing. FFT by 
comparison is a global trmisform approach that provides results with some delay. 

Audio signal processing systems fonn the basis for signal enhancement and noise reduction 
20 in a variety of voice related applications, such as speech recognition, speaker recognition, 
and voice synthesis. The pmsent scheme using a resonator array serving as dynamic filter 
array provides a powerful architecture for the implementation of algorithms that closely 
mateh signal processing in natural hearing and speaking. This property is considered to be a 
major innovative step leading to a new class of Audio signal processing systems (audio 
25 processors) that improve the overall perfonnance of future voice products. For example, 
speech recognition relies on a front end that transforms the voice signal into a set of 
descriptors (termed cues) of the properties of the signal. The quality of the cues is 
extremely critical for the robustness of the system in terms of speaker and enviromnental 
dependencies. The results that can be obtained with current systems arc far firom 
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satisfactory* The dynamic filter array, according to the present invention, allows to 
construct efficient algorithms for generating better descriptor sets» in particular also with 
respect to noisy environments, e.g,, voice computer communication in cars. 

The present scheme can also be used for processing of signals other than audio or voice 
5 signals. In other words, the present scheme can be used in many different lands of signal 
processing systems. 

According to the present invention, an input signal A(t) is received via an input line 1 and 
analyzed by a signal processing system 2 using a (damped) resonator array 9, as 
schematically illustrated in Figure I A (note that there can be more than one input signal, as 
10 shown in Rgure IC for example). The simplest form of a resonator is that of a damped 
harmonic oscillator, which follows the differential equation 

^ = ~wm) - uim -Am (d 

where Bi(t) is the response of the i-th resonator and A(t) is the input signal (e.g., a voice 
input sig?ial), Wi are the resonance frequency of the i-th resonator and tf is the 
15 corresponding decay constant. The Q-factor is defined by g = 

The i^onator can be implemented as an algorithm for digital computers using Verlet 
algorithm, for example, to solve the differential equation for the resonators of array 9. 

Bia+A) = n^B.-(0-^fi.a-A) + T^U(r+A)-A(r-A)] (2) 

Where A is the discretization time step. 
20 The frequency response of the damped harmonic oscillator is 

_ ft . SL\\''^ (3) 
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It is normalized to 1 at the resonance frequency, where the absolute value of the response is 
maximum. For w » Wi the amplitude response decays as — and for w « xoi the 
amplitude resiK)nse decays as 

The response of an array of resonators 9 is weighted with factors (weights W0i,t)\ that 
5 may depend on the resonator, i.e. its frequency /• = mtJln, and the time t. The weight 

factors may be calculated using some formalism from the actual input signal A(t) in order to 
respond to the instantaneous environment As an example one might exploit the quasi 
periodic nature of vowels, by using a pitch dependent, oscillatory amplification, to enhance 
certain speakers above the noise level The weighting of signals is schematically illustrated 
10 by means of weighting units 3, 4, and 5, These weighting units 3-5 are part of a so-called 
reconstructor 8. 

The weighting may be such that characteristic speech sounds are optimally separated from 
others and from background noise. Using the weighted response may directly identify 
different vowels and classes of consonants. 

15 Likewise, the weights can be constant. Under certain circumstances one or more of the 
weights might even be equal to 1, i.e. - L 

A combined signal C{t) is composed by superposing the / weighted resonator signals. In the 
following equation the superposition is a summation of the f weighted resonator signals. 

C(0-SW/(OBi(0- ^"^^ 

i 

20 The superposition is done by a superposition circuit 6 that is part of the reconstructor 8. 

Note that in the present context the word superposition is used to describe any 
mathematical operation (adding, subtracting, multiplying, dividing, and so on) that is 
employed to process the i weighted resonator signals such that M output signals C(t) are 
obtained, where M € M (in other words W = U 2, .,.). Preferably, M<N. 
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Another embodiment is illustrated in Figure IB. An input signal A(t) is received via an input 
line 1 i and analyzed by a signal processing system 12 using a resonator array 13. The N 
individual output signals fi/frA Bz(t), BN(t) of the array of resonators 13 are forwarded 
to a reconstructor 15. This reconstructor assigns an individual weight Wi(t^fi} to each of 
5 flie N individual output signals Bi(t), B2(t), BN(t) to generate N individual weighted 
output signals. Furthermore, the reconstructor 15 provides for a superposition of these N 
individual weighted output signals to generate M output signals Ci(t), Czit), C^it) with 
ikf€f^ae..M=l,2,...). 

The invention lies in the fact to replace traditional signal processing approaches, which are 
10 based for example on Fourier or Laplace transforms, directly by the response of a 

differential equation, that selects certain spectral features. The advantage of the invention 
lies in its simplicity, its flexibility with inspect to spectral resolution, which may itself 
depend on the frequency region, and in the nearly eliminated time lag between input signal 
A(t) and resonator response B(t), 

1 5 This present methodology can be generalized to resonators having arbitrary response 
functions using other differential equations for the resonator response or their 
corresponding physical implementation. The differential equations are not limited to second 
order. 

The resonator array may comprise N Lorentzian resonators, or any combmation or 
20 arrangement of K Lorentzian resonators that are connected or coupled so as to form N 
resonators, or N non-Lorent^ian resonators, or any combination or arr^igenaent of K 
non-Lorentzian resonators that are connected or coupled so as to form N resonators, or any 
combsnaaon of Lorentzian and non-Lorentzian resonators so as to form N resonators (with 
mdK^Ne N). Please note that the resonators can be implemented in hardware, 
25 software, or a combination of hardware and software. 

In known FFT-based signal analysis the response time is given by the lowest resolved 
frequency for all Fourier components. With the present scheme, the response time is given 
directly by the decay time r,» which can be selected at will and for each resonator 
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individually. Thus high time resolution can be achieved in the high frequency region, which 
is less relevant for vowels but which i& important for a number of consonants, for which 
time resolution is crucial. 

In order to be able to design an efficient system, or a system that is well suited for a 
5 particular application, there is some time dependency introduced. In other words, at least 
one of the parameters (i.e. the resonance frequency, or the weight, or the Q-factor, or the 
like) describing or characterizing the ptcsent signal processing system are time dependent. 
This is schematically illustrated in Figute IC where there are L time signals P(t), with 
L € N (Le, 1=1, 2, ...). These L time signals P(t) are fed to a signal processing system 16 
iO which comprises a resonator array 17 and a reconstmctor 18. Please note that the system 
of Figure IC is more general than the systems 2 and 12 of Figures lA and IB because there 
ate V input signals A(t) with VeN (Le, V=l, 2, These Vinput signals can be 
independent. It is for example conceivable that there are V microphones each of which 
provides an input signal Aiffj to the system 16. 

15 There are several options to generate or derive the L time signals P(t). These signals can be 
generated or derived fifom 

- the input signal(s) A{ t) (in a feed forward fashion), 

- the M output signals C(t) (in a feedback fashion), 

- any signal inside the signal processing systems 2, 12, or 16, 
20 - another signal processing system, 

- another reconstmctor, 

- another resonator array, or 



- a computer system. 
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One possible software implementation of a general damped (harmonic) resonator array is 
now described by means of a pseudo-code representation which is illustrated in Figure 13A 
and 13B. These Figures comprise comments in brackets [ J, 

This software implementation of a general damped (harmcHiic) resonator array comprises a 
5 first loop (this loop starts at reference number 130 and finishes at reference number 13 1) 
that is used to describe the behavior and properties of each of the resonators of a 
resonator array and the weights weight(N). At the begmning of this loop omega(N) (=sf2 n), 
the tune decay constant tau(Nl and weight(N) of the respective resonators are fetched, 
nien a set of constants are calculated that allow to represent Equation (2) in the form 

10 Biit + A) ^ uoMt) + UmM^ - A) + Ui, [Ait + A) - A(/ - A)] (2a) 

that describes the behavior of the respective resonators. At the end of this loop 130-131 the 
amplitudes bm and bo are initialized to zero. Note that the Equation (2a) is actually solved 
farther down (indicated by reference number 134 in Figure 13B) in the pSeudo-code and 
not inside the first loop 1 30-13 L 

15 This first loop 130-131 is foUowed by a loop 132433 over time steps ti. The stack for input 
signal a (at reference number 135) is reset when nmning through this loop 132433 for the 
first time. This is necessary, because the stack 135 would ofterwise be undefined. Then the 
difference cprime of the input signal a at two time steps is determined. The resonator 
amplitudes Bi are depending on the actual value of aprtme. 

20 If the weights {weight(N)) are constant, then the output signal c can be calculated right 
away by weighing and superposing the individual resonator output signals. If the weights 
{weighm) are not constant, then they have to be calculated before weighting according to 
same procedure and superposing the individual resonator output signals. Note that in the 
present software implementation die individual resonator output signals arc superposed by 

25 means of adding them. 
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The underlying scheme can be further generalized, as illustrated in the schematic flow chart 
given in Figure 2. In a first step (box 20), an input signal Af/) is received. This input signal 
A(t) is then processed by AT resonators such that iV individual output signals Bi(t), B^t), 
Bi4t) are generated (box 21). These //individual output signals Bi{t), Biit), Bi^t) are 

5 weighted using a corresponding weight Wi(t), W2(t), Wivf/J for each output signal to 
generate N individual weighted ouQ)ut signals, as indicated in box 22, As a result of this 
step the following N individual weighted output signals are obtained: Bi(t)Wi(th B2(t)W2(t), 
Bf4t)WN(t). The weighting of signals is schematically illustrated by means of boxes 3, 4, 
and 5. Finally, the N indiAddual weighted output signals are superposed by a superposition 

10 citcuit 6 to generate M output signals Ci(tl Ciit), d^t), wiA MsH (ie, M=l, 2, ...) 
(see box 23). Please note that in Figure 1 A there is just one output signal C(i) provided at 
output line 7, i.e», in this implementation example M = L 

The generation of the N output signals BM BiUh B^lt) can either be done concurrently 
(i.e, in parallel), or sequentially. A ujechanical resonator array witti N cantilever resonators 

15 will process tiie input signal A(t) concurrently to generate the N output signal B;ff), B2(t), 
BMt)f whereas a computer program implementation (such as the one described by means 
of pseudo-code) may generate the N output signals Bi(t), BzUh SjvCO one after the other, 
i.e. sequentially. In the above pseudo-code example this is done by means of a loop which 
is run through times. If the output signals BM B2(th BMO are generated 

20 sequentially, the respective output signals have to be stored (either before the wei^g step 
or afterwards) until all output signals dxt available for supeiposition. 

Now the architecture of an audio processor embodiment, based on the resonator transform 
method, is described. In particular, the present scheme ts exemplified using an adaptive 
digital hearing aid as a working embodiment. 

25 At high frequencies the frequency resolution of the human ear is lower than at low 
frequencies. Correspondingly also the frequency resolution, and with it the spacing of 
resonators of a resonator array, can be matched accordingly, saving computational 
resources, for example. Realization of such an unequal spacing in the frequency domain is 
nontriviai in standard approaches. 
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In the following section the fr^uency-dependent signal amplification of hearing aids is 
described. Hearing aids amplify cer^n spectral regions. Existing technology uses Fourier 
transforms of die input signal on short time intervals. This technique incoq>orates a fixed 
time lag between input signal and the synthesized output signal, which is inversely 
5 proportional to the frequency resolution. Fourier transformation requires inherently to 
compromise frequency information with time information. This is problematic because 
speech exploits bodi extreme time and frequency resolution simultaneously* The amplified 
signal is synthesized in the traditional approach from scratch using the Fourier back 
transfonnation. 

10 According to the present invention, the audio input signal A(t) is only enhanced using a 
resonator transformation, and the frequency information such as the pitch remains fully 
intact, because the resonator oscillates with the same frequency as the audio input signal 
A(t). This approach mimics to some extent the human hearing apparatus, Le. the cochlea, 
which can be looked upon as a complex resonator array. 

15 Hearing impairment is characterized by elevated thresholds for the detection of sounds in 
quiet A typical example for the hearing threshold curve 30 of an impaired ear is shown in 
Figure 3. A prominent characteristic is the steep increase of the hearing threshold above 
200 Hz with a first maximum of 50 dB at 1 kHz (reference number 3 1) and a further 
increase of the threshold in the 10 kHz range. Despite the loss in sensitivity, a sound at high 

20 intensity might soimd equally loud to a hearing impaired listener as it does to a normally 
tearing listener. In other words, there is an abnormally steep growth of loudness with 
intensity in the impaired ear. This phenomenon, called recruitment, is illustrated in Figure 4. 
Because of recruitment, restitution of impaired hearing cannot be accomplished by means 
of a simple linear filter which compensates the loss of sensitivity of the ear. In order to 

25 preserve a natural perception of loudness, amplification of the sound intensity at a 

particular frequency must be adjusted according to the hearing threshold and according to 
the momentary intensity at this frequency. This process is illustrated in Figure 5. In panel 
(a), a white noise input spectrum with 0 dB intensity is considered, meaning that the noise 
is just at the detection threshold for a normal hearing person. Note that the curves on the 
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left hand side of Hgure 5 represent the input intensity in dB, whereas the curves on the 
right hand side represent the characteristics of a hearing impaired person (amplitudes in 
dB). In order to create the same acoustical sensation for a person with hearing impaiiment 
the noise spectrum must be amplified by the same amount as the hearing threshold (see 
5 Figure 4) increases at a given frequency. Using these amplification factors for an input 
noise level of 50 dB (see panel (a)), which corresponds to the loudness of a normal 
conversation, the sound pressure at the impaired car would be in excess of 100 dB at 
frequencies above 1 kHz. 

Due to recruitment, such a sound level is perceived as extremely loud if not painful also by 
10 a person with hearing problems. To create a balanced acoustic sensation the amplification 
curve may be scaled for example such that the perceived loudness level is 60 dB for all 
frequencies. For the example shown m Figure 3, the hearing threshold at 1 kHz is at about 
60 dB which corresponds to the situation depicted in Figure 4. From the dashed curve 41 in 
the Figure 4 one can conclude that the physiologically correct sound pressure is 80 dB* In 
15 other words, an amplification of 30 dB is sufficient as opposed to Ao = 60 dB for low level 
signals. It follows that the scaling transformation for hnear recruitment is a multiplicative 
compressicHi of the ampUfication curve according to 

A=Ao(l-P/Pnm) (5) 

where P is the sound pressure of the input signal and Fmax the intercept point 40 (approx. 
20 100 dB, see Figure 4) for equal loudness perception of impaired and normal hearing. The 
spectral distribution of real sound signals (curve 52) 52 is not uniform, however (see panel 
(c) in Rgure 5). Accordingly, the scaling factor varies with frequency. Correspondingly, the 
shape of the amplification curve 53 which is adjusted dynamically can deviate substantially 
from the hearing threshold curve 30 (see Figure 3). 

25 The present resonator transform method is ideally suited for implementing acoustical signal 
processors as it provides a natural path for implementing functionality which closely 
resembles the way in which the cochlea of the ear operates. It is to be emphasized at this 
point that the cochlea does not perform a sharp Fourier analysis of sound signals. It is, in 
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fact, more appropriate to visualize ttie cochlea as a bank of broad band filters which can be 
roughly approximated by Lorentzian resonators with a Q-factor of the order of 3 to 5. The 
loudness perception is proportional to the logarithm of the physical intensity and thus an 
extremely large dynamic range can be processed by the ear. In order to cope with this 
5 property, signals should be represented by at least 16-bit numbers, if the present invention 
is realized in a computer system. It is also important to realize that loudness fluctuations by 
tens of dB occur within milliseconds in natural sounds which calls for a rapidly responding 
system* Hence, it is advantageous to employ low Q-factor resonators in an audio processor. 
For this reason, the Q-factor of the individual lesonatCMS should be below 100 and 
10 preferably below 10. A suited range for the Q-factor is between 1 and 5. 

Figure 6 describes a hearing aid embodiment A block diagram of the signal processing 
system 60 diat is part of the hearing aid is shown in Figure 6. The input data A(t) is a 
discrete stream of 16-bit samples taken at a rate of/, = 44, 1 kHz. The resonator array 63 
splits the mcoming signal AU) into A/^ resonator transforms Bt{t), Depending on the actual 

15 value of the resonator transforms the weights Wtit) for the resonator transforms are 

calculated, which are multiplied with the resonator transforms in the devices denoted with 
64.1 - 64.N before the weighted resonator tr^sforms (heiehx referred to as individual 
weighted output signals) are combmed in fte superposition circuit 65 to form the output 
signal Qt). In the present embodiment two steps are involved in determining the weights 

20 Wit). The first step (calculation of filter function) detennines the relevant sound pressure in 
each resonator transform, and from that the optimum amplification at this frequency. The 
calculation of the filter function is carried out by units 6L I - 61,N, Then a filter function is 
synthesized by an approximation cucuit 62 to produce an amplification as given by the 
instantaneous frequency dependent sound pressure <I(a>i)>. For this purpose, the 

25 approximation circuit 62 evaluates the weights WM according to Equation (8). Units 
61.1-61.N and 62 serve the dynamic calculation of Wit). 

In the embodiment of Figure 6 the resonator transforms Bi{t) are calculated by means of a 
numerical integration of a generic differential equation similar or equal to the one given as 
Equation (I). 
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The resonator transform coiresponds to a filtering of the input signal A(t) with the 
resonator transfer function being given by Equation (3). 

In the present embodiment of the hearing aid (see Figure 6), 15 resonator transforms are 
used. They have a constant Q-factw of 3 and their resonance frequencies tOi are 
5 logarithmically spaced over two decades between 1(X> and lO'OOO Hz. The output signal 
C(t) is a linear superposition of the resonator transforms 

C(f)=IW;(OS,(/) (6) 

i 

where the weights Wi(t) are teal numbers. They are calculated such that the desired transfer 
characteristics of the signal processing system 60 is obtained. Calculation of the weights 

10 Wi{f) involves two steps. Firstly, in each sampling interval the momentary filter function is 
determined by circuits 61 . 1, 61.2, 6LN, and secondly^ the desired filter function is 
approximated in an approximation circuit 62 by the resonator response functions using a 
least squares fit procedure* Note that the circuits and other elements of Figure 6 naay 
represent a software implementation as well as a hardware implementation of the present 

15 scheme. 

Calculation of the filto function by circuits 61.1, 61.2, 61.N is described in coimection 
with Figure 7 which shows exemplary elements of one such unit 61 i. For die evaluation of 
the filter function a measure of the sound pressure in each resonator transform is 
determined. This measure should on the one hand reflect the average sound pressure over a 

20 period of some seconds. On the other hand sudden rises in sound pressure must be detected 
so that the system can shut off amplification quickly to avoid damage to the ear. For this 
purpose, the resonator transform B,(t) is processed by unit 70 to obtain the intensity signal 
Then, the signal lit) is split into two signals at 7 L One of the signals U t) is processed 
though a resonator 72 that has non-zero damping with a r in the range of seconds. This 

25 resonator 72 is mlatively slow damped resonator which avemges the signal over time. The 
logarithm of the result is taken by a unit 73. The logarithm of the second part of the signal 
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W) - as generated by unit 74 - is transformed using a similar resonator 75 with zero 
frequency and a damping in the range of milliseconds. The second resonator 75 is faster 
than the first resonator 72. The two processed branchy 76.1 and 76.2 are combined into a 
measure <I> of the relevant sound pressure at the frequency of tiie resonator transform. 
5 The combining of signals is done by unit 77. This sound pressure is then used to determine 
the filter curve at the given frequency adjusted to the sound pressure according to Equation 
(7). This is done by unit 78. The ontpnt line 79 is connected to the approximation circuit 
62. 

Hie intensity of the i-th resonator response h (t) - Bf{t) corresponds to the spectral power 
10 of the input signal A(t) at Wi averaged over a frequency band of width Wi/Q. For each 

resonator an averaged logarithmic spectral intensity, </ is calculated which is defined as 
the maximum of the logarithm of a slow running average of li ( t) with a time constant 
Xs = 0.5s (resonator 72) and a fast running average (resonator 75) of the logarithm of It (t) 
with a time constant t/ = 5ms for Wi > 2n • AOOHz and t/ - 5ms • In • AQOHzlrDi for 
15 tj3i<2rt< 4Q0Hz (note that taking the logarithm of the average and taking the average of 
the logarithm are inequivalent operations). The mnning averages are obtained by means of 
a recureive summation algorithm in which the output of step n+i is set equal to the input 
multiplied by a weight l/Ns/ plus the output of step n multiplied by a wei^t Ns/- VNs^ 
where Ns/^f^Zsj^ is the number of steps over which the averaging is performed. The 
20 intensity algorithm is designed tp follow fast intensity peaks quasi instaiitaneously while at 
the same time it also provides a measure of the average sound leveL In accordance with 
Equation (5) the transfer function F(iz7i)of the adaptive filter sampled at Wi is derived from 
the spectral intensities </,> using the transformation 

25 where FoiWi) is the hearing threshold of the impaired ear at Wh Thus recruitment is 
individually accounted for in each frequency band that is sampled by the corresponding 
resonator. 
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The desired filter transfer ftmction is defined only at the resonator eigen-firequencies n?,*- 
The complete transfer function is interpolated using a linear superposition of the resonator 
response functions and imposing the minimization criterion 



-F{tDj)y-sxm (8) 



5 The minimization problem is solved iteratively using a standard Newton algorithm whereby 
one iteration step is performed per sampling interval. Note that the filter function is 
continuously updated during iteration. In each iteration step a new set of weights d is 
computed which is used for the synthesis of the output signal (see Equation 6). 

Hie above hearing aid was tested with a test person from whom the hearing threshold 
10 curve shown in Figure 3 was available. The overall performance was very positive. In 
particular, comprehensibility of natural speech was better than with the test person's 
conventional hearing aid* The test person was able to hear sounds that were not known to 
him before. This is an indication that the high frequency gain of standard hearing aids is 
kept rather low on purpose in order to suppress painfully loud sound levels at the ear, 

15 An alternative implementation of the adaptive filter (a pseudo-orthogonal basis function 
filter 80) is shown in Figure 8. In this Figure an implementation of a hearing aid using 
nearly non-overlaping frequency responses is implemented. By suitable superposition of the 
iV resonator transforms in a mixing matrix 83 with weights Afijj, AT new resonator 
transforms are constructed, which have a narrower frequency response. The Lorentzian 

20 resonators 82 together with the mixing matrix 83 form a non-Lorent^ian resonator array. 
As each resulting resonator transform Bi(() (set of quasi-orthogonal basis functions) is 
localized in a narrow frequency domain, the coefficients O in Equation (8) can be taken to 
be F(Wi). Hence, only the filter function needs to be calculated by units 8L1-81,N and the 
synthesis of the filter function is just an identity transform. 



25 



The basis functions are linear superpositions of resonator transforms 
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Bdt)=iAijBjit) (9) 

and they are constructed such that they have a Lors^tzian line shape at &eir resonance 
frequencies ©/and such that the mutual spectral overlap can be neglected for the purpose of 

5 the filter synthesis, as indicated above. As a result, approximation of the filter transfer 
function by circuitry 81.1, 81.2, 8LN is straight forward. The output signal C(?) is a 
linear superposition (performed by superposition unit 84) of the B/(f)with the weights now 
being equal to the desired amplification of the filter, at the corresponding resonance 
frequency. The weights F(&i) are computed by the same method as in the embodiment that 

10 is illustrated in Figure 7* 

We use TV = 10 basis functions to implement die adaptive filter for the hearing aid of Rgure 
8. The center frequencies of the basis functions are again logarithmically spaced over a 
frequency range from 100 Bz to 10 kHz. The characteristics of the basis functions are as 
follows: They have a Loientzian line shape with a Q-factor of 3 in a frequency segment of 

15 m± m /Q, a very steep gain roll-off of 100 dB/decade on the low frequency side, and a 
moderate gain roll-off of 40 dB/decade on the high frequency side. Hie aggressive low 
frequency attenuation is needed in order to cope with the steep gain profile of the hearing 
curve between 200 Hz and 1 kHz which corresponds to a slope of 80 dB per decade. To 
meet the demanding requirements a large set of Af = 100 resonator transforms is needed for 

20 the synthesis of the basis functions. Simulations show that a value of 3 is a good choice for 
fee Q-factor of the resonator transforms. The corresponding resonance frequencies are 
logarithmically distributed between 100 Hz and 10 kHz in analogy to the basis functions. 

There aie various ways to couple the hearing aids, as described above, with the human ear. 
One approach is illustrated in Figure 9. As shown in this Figure, the hearing aid comprises a 
25 signal processing system 9 1 , according to the present invention, which receives an input 
signal A(t) (e.g., an acoustic signal) via input 92. The signal processing system 91 may 
comprise a signal processing system 2 or 12, as illustrated in Figures 1 A and IB, 
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respectively. The output signal C(t) is fed via output 93 to an interface circuitry 90. This 
interface circuitry 90 provides an interface between the signal processing system 91 and a 
human being. In the present embodiment this interface circuitry 90 comprises an amplifier 
94 which amplifies the signal C(t) and a loudspeaker 95. The loudspeaker 95 is designed 
5 such that it can be either placed tight next to the ear or within the ear. The loudspeaker 95 
feeds an acoustic signal into the outer ear. 

Anoflier approach is illustrated in Figure 10. As shown in this Figure, there is an interface 
circuitry 100 that interacts directly with the human body. The interface unit 100 receives 
via input line 102 M output signals C!(t), C2(t), .»> CwCt) from a signal processing system 

10 according to the present invention ( where M e N). The signal lines 101 can be connected 
to special implants 102 that are situated in the inner part of the ear. Signals are fed via the 
signal lines 101 to the implants 103 such that certain segments in the ear 104 are 
stimulated. The stimulation of the ear by means of implants 103 is used if a person is deaf 
or partially deaf. Nowadays, in certain cases electrodes are even implanted into the inner 

15 ear directly contacting the nerves of the ear. In this case, the interface circuitry 100 has to 
be adopted such that the signals on signal lines 101 are suited to stimulate the nerves 
cSrectly. For tiiis purpose, the interface circuitry 100 may comprise a processor which 
processes the input signal C(t) so as to generate electrical pulses which can be directly fed 
to the implanted electrodes. The present invention helps to improve current hearing aids 

20 because it provides a more reliable and more natural way to process input signals. 

The interface circuitry may also be used to connect signal processing systems according to 
the present invention to a back end system, such as computer, telephone system, hearing 
aid, speech recognition system, speaker recognition system, or a pervasive computing 
device, for example. The interface circuitry can be realized by means of discrete elements, 
25 or it can be integrated into a chip. The interface circuitry may comprise a microprocessor, 
multiplexers/demultiplexers, parailcl-to-serial converters, serial-to-parallel converters, 
analog/digital conversion circuits, memory, and so forth- 

In the present embodiment of the hearing aid, the orthogonal basis function approach is 
numerically less efficient than the direct resonator transform method even though no 
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iterative algorithms are involved. However, in applications with relaxed attenuation 
requirements the basis functions can be synthesized with an accordingly smaller number of 
resonator transforms, making the orthogonal basis function method eventually the preferred 
embodiment* 

5 The present signal processing systems can also be used as part of speech recognition 

systems. These speech recognition systems are used to simplify input of commands or text 
into computer systems, for example. Also handicapped persons rely more and more on 
technical and electrical apparatus which can be operated by giving acoustic commands. 
Furthermore, pilots, car- or truck-drivers, technicians, and surgeons will use the powerfQl 

1 0 and reliable speech r^ognition systems enabled by the present invention. 

A speech recognition system according to the present invention relies to a certain extent on 
a set of descriptors (termed cues) output by the inventive signal processing system. This 
signal processing system is employed as front end that transforms a voice signal (input 
signal A(t}) into the set of descriptors. This set of descriptors describes certain 
15 characteristic properties of the voice signal (input signal A{t)). The set of descriptors can be 
compared with information stored in a knowledge database or a training database. If a 
matching set of descriptors of found in the database, then the corresponding, vowel, 
consonant, syllable, or word might be returned for further processing. The database of such 
a speech recognition sj^tem can be much smaller than the one of known systems. 

20 The present signal processing systems is well suited for use in speaker recognition systems. 
A speaker recognition system is a system which is used to identify a speaker by its voice. 
This is important for systems where some identification of the user is required. Examples 
are access systems, automated teller machines, and the like. 

There is also a demand for acoustic detectors which are designed to detect a particular 
25 noise or sound. Such a detector could for example be used to indicate whether an engine is 
about to be destroyed, or to detect acoustic signals which can otherwise not be detected by 
the human ear. In noisy environment. e,g, in a cockpit, it would be useful to reduce or 
ehminate noise so as to ensure that voice and other signals can be better understood. Such 
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a detector has to be able to process noise input A(t} in order to generate an output signal 
C(t) which is fed to a loudspeaker for the suppression of the noise input For this purpose, 
the ou^t signal C(t) has to be phase-shifted befoi« it is fed to the loudspeaker. The 
supeiposition of the output signal C(t) and the noise input Affj leads to a reduction in the 
5 overall noise. The present invention is well suited for such systems, because it can be 

designed or adjusted to process and suppress special noise, such as the noise from tires, for 
example. 

Another damped electric resonator 1 10 is shown in Figure 1 L This resonator 1 10 comprise 
a resistor Rp, a capacitor C, and a coil L. The resonance frequency of this damped 
10 resonator 1 10 is: 



One can use N such resonators 1 10 to build a resonator array 120, as schematically 
15 illustrated in Rgure 12. Preferably, each resonator of the resonator array 102 has a different 
Q-factor and resonance frequency. In Figure 12 there are N individual output signals B\(t), 
Bzith .... BMt). Each of these individual output signals B}(th B:^th BN(t) is then 
forwarded to a reconstructor 121. This reconstructor provides for the weighting and 
superposition of the signals such that M output signals Off j are obtained. 

20 As mentioned farther above, the present scheme cm be used for the processing of any kind 
of signals. If a signal processing system according to the present invention is designed to 
operate on eJecuical signals and if an acoustic signal is received, then some sort of 
microphone of other means for transformation of the acoustic signal into an electrical signal 
format is required. A uiicrophone is an eiectroacoustic transducer which converts variations 

25 in sound pressure into an equivalent etecu-ical signal, which then is fed to the signal 




(10) 



where is the resonance frequency without damping, and the Q-factor is 




(11) 
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processing systent Any microphone is well suited for use in connection with a signal 
processing system acpording to the present invention. 

The present invention can be realized in hardware, software, or a combination of hardware 
and software. Any kind of computer system or other apparatus adapted for carrying out the 

5 methods described herein is suited. A typical combination of hardware and software could 
be a general purpose computer system widi a computer program that, when being loaded 
and executed, controls the computer system such that it carries out the ix^thods described 
herein. The present invention can also be embedded in a computer program product, which 
comprises all the features enabling the implementation of the methods described herein, and 

10 which - when loaded in a computer system - is able to cany out these methods. 

A computer program or computer program means in the present context any expression^ in 
any language, code or notation, of a set of instructions intended to oiuse a system having 
an information processing capability to perform a particular function either directly or after 
either or both of the following a) conversion to another language, code ot notation; b) 
15 reproduction in a different material form. 
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CLAIMS 

1 . A signal pxocessing system for processing an input Signal, comprising: 

a plurality of resonators, each resonator having parameters characterizing it, for 
processing the input signal to generate N individual output signals; and 

5 a reconstructor 

for weighting each of the iV individual cmtput signals using a corresponding weight 
to generate individual weighted signals; and 

for superposing the iV individual weighted signals to obtain M output signals; 

whereby A^^ € and M^M^ and whereby one of the parameters or the weight is time 
10 dependent. 

2. The signal processing system of claim 1, wherein one of the iV^ resonators is a 
Lorentzian resonator. 

3. The signal processing system of claim 1 » wherein one or more of the plurality of 
resonators are Lorentzian resonators, and wherein the AT individual output signals are 

15 fed to a mixing matrix for coupling of these N individual ou^ut signals, 

4. The signal processing system of claim 1 , wherein the plurality of resonators form a 
dynamic filter array. 

5. The signal processing system of claim 1, wherein the weight is time dependent and 
frequency dependent, 

20 6. The signal processing system of claim 1 , wherein the reconstructor comprises circuitry 
for dynaxnically calculating die corresponding weights used for weighting each of the 
N individual output signals. 
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7. The signal prcx:essing system of claim 1, wherein the reconstructor superposes the N 
individual weighted signals by a mathematical operation selected from a set of 
operations consisting of addition, subtraction, multiplicaticoi, division* 

8. The signal processing system of claim U wh^ein the reconstructor comprises an adder 
5 for superposing the N individual weighted signals. 

9. The signal processing system of claim 1 further comprising an interface circuitry 
serving as interface to a back end system. 

10. Th& signal processing system of claim 1, wherein the back end system is a hearing aid, 
or a speech recognition system, or a speaker recognition system, or a pervasive 

10 computing device, or a computer system. 

1 1 . The signal processing system of claim 9, wherein the interface citt:uitry comprises an 
amplifier and loudspeaker. 

1 2. The signal processing system of claim 1 , wherein one of the plurality of resonators 
comprises a coil, a resistor, and a capacitor. 

15 13. The signal processing system of claim 1, wherein one of the plurality of resonators 
comprises a micromechanical element, preferably a cantilever 

14. The signal processing system of claim 1 , wherein one of the plurality of resonators is 
realized by a combination of a processor and a code that embodies an algorithm for 
execution by the processor. 

20 15. The signal processing system of claim 1 , wherein the plurality of resonators and the 
reconstructor are realized in a computer system comprising a processor and a code 
that when executed by the processor generates the M output signals. 



16. 



The signal processing system of claim i, 9, 10, or 15, wherein the input signal is a 
voice signal. 
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17. TTie signal processing system of claim 1 or 15, wherein the M output signals represent 
a set of descriptors which describe properties of the input signal. 

1 8. The signal processing system of claim 17, wherein the set of descriptors is usable by a 
speech recognition system or speaker recognition system. 

5 19. The signal processing system of claim 1 or 15, wherein the iNT individual weighted 
signals identify different vowels and/or classes of consonants. 

20. The signal processing system of claim 15, wherein the code embodies an algorithms 
for generating a descriptor sets for use in a speech recognition system or speaker 
recognition system. 

10 2U The signal processing system of claim 14 or 20, wherein the algorithm represents a 
diferential equation, preferably a second order equation. 

22. The signal processing system of claim 1 , wherein there is more than one input signal. 

23. The signal processing system of claim 1 receiving L time signak which either make the 
parameters or the weights time dependent and wherein Z. e N. 

1 5 24. The signal pmcessing system of claim 23, wherein the L tfane signals are provided by a 
resonator, a reconstructor, or a computer, 

25. Method for processing an input signal, comprising the steps: 

processing the input signal by means of a plurality of resonators, each resonator 
having parameters characterizing it, to generate N individual output signals; 

20 weighting each of the individual output signals using a corresponding weight to 

generate N individual weighted signals; and 

superposing the N individual weighted signals to obtain M output signals; 
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whereby N&N and M e and whereby one of the parameters or the corresponding 
weight is time dependent 

26. A computer program element comprising: 

computer program code means to make a computer system execute procedure for the 
5 processing of an input signal by: 

processing the input signal to generate N individual output signals; 

weighting each of the N individual output signals using a corresponding weight to 
generate individual weighted signals; and 

superposing the N individual weighted signals to obtain M output signals; 

1 0 whereby either for the processing of the input signal or the weighting of each of the N 

individual output signals the representation or equivalent of a time signal is used, 

27. A computer program product for the processing of an input signal, said computer 
program product comprising a computer readable medium, having theieon: 

computer program code means, when said program is loaded, to make a computer 
1 5 system, execute procedure to: 

process the input signal to generate N individual output signals; 

weigh each of the N individual output signals using a corresponding weight to 
generate iV individual weighted signals; and 

superpose the N individual weighted signals to obtain M output signals; 

20 whereby either for the processing of the input signal or the weighting of each of the ^ 

individual output signals the representation or equivalent of a time signal is used. 
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ABSTRACT 



Schene for processing an input signal A(t) by N resonators (17), each having parameters 
characterizing it, to generate JV individual output signals. Then each of the individual 
output signals is weighted using a corresponding weight to generate AT individual weighted 
5 output signals which are superposed to obtain M output signals C{ t). One of the parameters 
CM- the weight depend on a time signal P( t). 



10 



15 



20 (Figure IC) 
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FIG, 13A 

Implementation exatt^le: 

tread discretization time step] 
get dt 

[reset oscillators] 

for each N [loop over resonators] 

get omega (N) [read frequency *2*pi] 
get tau(N) [read time decay constant) 
get weight (N) [read weight] 

Uom)= (2-(omega{N)dt)^) / (l+dt/tau(N)) ^ 

(one example of equations describing 
[a damped harmonic resonator] 

u„<N)t= -{1 - dt/tau(N)) / (l+dt/tau(N) ) 

Ui(N)- dt / (1+dt/taum) ) / tau(N) 

[u are constants} 

hm(Ni=0 tb are amplitudes of resonators] 

bO N -0 tbm is amplitude of previous time step] 

[i>o is amplitude of actual time step} 

done with loop over N 
[loop over time steps] 

for each t . , ^ 

input ap tread input signal a of the] 

[next time step] 

if (first t) then [the first time, reset stack] 

ix^ciir&u u; i^x* ^^^^ input; otherwise a would] 

[not be defined] 

ao=ap 
a»=^ap 
end if 

[calculate difference aprime of input signal a] 
aprime = (ap-am) 



continue with FIG. 13B 
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HG. 13B 

continuation of FIG. 13A 

{propagate resonators; loop over all resonators! 
for each N 

bp uo{N)*bo(N) + u«(N)*h.<N) + Ui (NT) *apriine 

[resonator for following time] 
[step] 

b»(N) = bo(N) [stack is updated/ shifted] 

bo{N)^ ^ bp 
done with loop over N 



[at this point the weights weight (N) can be recalculated] 
[as a function of the bo{N) 1 

[In this implementation example the weights are constant] 

[weight and combine resonator output signals to generate] 
[output signal cl 

c=0 

for each N 

c = c + weight (N) *bo{N) 
done with loop over N 

[reset stacJc for input] 
a* =^ ao 
ao - ap 

[output output signal c] 

output c 

done with loop over t 



end. 
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